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Abstract 

The software correlator of the Chinese VLBI Network (CVN) has played an irreplaceable role 
in the CVN routine data processing, e.g., in the Chinese lunar exploration project. This correlator 
will be upgraded to process geodetic and astronomical observation data. In the future, with several 
new stations joining the network, CVN will carry out crustal movement observations, quick UT1 
measurements, astrophysical observations, and deep space exploration activities. For the geodetic or 
astronomical observations, we need a wide-band 10-station correlator. For spacecraft tracking, a real- 
time and highly reliable correlator is essential. To meet the scientific and navigation requirements 
of CVN, two parallel software correlators in the multiprocessor environments are under development. 
A high speed, 10-station prototype correlator using the mixed Pthreads and MPI (Massage Passing 
Interface) parallel algorithm on a computer cluster platform is being developed. Another real-time 
software correlator for spacecraft tracking adopts the thread-parallel technology, and it runs on the SMP 
(Symmetric Multiple Processor) servers. Both correlators have the characteristic of flexible structure 
and scalability. 


1. Introduction 

At present, the Chinese VLBI network (CVN) consists of four stations (Shanghai, Yunnan, 
Urumqi, and Beijing) and one data processing center at Shanghai Astronomical Observatory (Fig- 
ure 1). CVN has successfully been used in the Chinese lunar exploration project (Chang’E) to 
track the CE-1 lunar probe. It will join the subsequent Chinese lunar and Martian exploration 
projects. Besides the deep space exploration activities, CVN will take more geodetic and astro- 
physical observations. For example, under the framework of the Chinese National Key Scientific 
Infrastructure Project ‘Crustal Movement Observation Network of China’ (CMONC), the CVN 
upgrade plan was approved in 2007. The purpose is to provide the primary fiducial points by 
performing geodetic VLBI observations monthly using the CVN antennas. Another project for 
rapid UT1 measurement on the Shanghai-Urumqi baseline was also approved last year. So far, 
the CVN pulsar observation is also under way [1, 2, 3]. 

The CVN Data Center now contains a 4-station software correlator and a hardware correlator. 
The software correlator has played an irreplaceable role in the CE-1 routine data processing. 
However, both correlators are specifically designed for the Chang’E project and do not meet the 
scientific data processing demands. After the CE-1 mission, we started upgrading the software 
correlator. It gradually obtained the geodetic and astrophysical data processing abilities. It was 
also used in all kinds of experiments, like the pulsar observations and the digital baseband converter 
test experiments. 

In the future, several new domestic antennas, including the new Shanghai 65-m antenna, will 
join in the CVN observations. Considering the possibility of using some Russian and Japanese 
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antennas, we can expand CVN to a local area VLBI network. In this VLBI network, the total 
antenna number will increase to 8-10, and the longest baseline will extend from 3000 km to 7000 
km (Figure 2). The expanded CVN is a multi-purpose VLBI network for geodesy, astronomy, and 
deep space explorations. It can be seen that a new CVN data processing center with a powerful 
correlator is badly needed. The software correlator is a good choice for such a medium-scale VLBI 
data center. 



Figure 1. Current CVN stations and data center. Figure 2. Extended CVN station map. 

The requirements on the CVN correlator for science and deep space navigation are different 
(Table 1). In general, there may be 2-10 stations in geodesy and astronomy observations. For 
example, we plan to use the Shanghai-Urumqi baseline to perform rapid UT1 measurements. But 
in astrophysical observations, 5-10 stations are needed to achieve good UV coverage. For tracking 
the first Chinese Martian deep space probe YH-1, we hope to organize 4-8 Chinese and Russian 
stations. Similarly, the data speed for geodesy and astrophysics is higher than 1 Gbps/station. 
But for most deep space applications, the observation data speed is less than 128 Mbps/station. 
However, the data turnover time and reliability requirement of the navigation observation is much 
higher than that of scientific observation. For deep space probe navigation, sometimes the total 
data latency of the VLBI system should be less than 1 minute and the correlator must have 
real time data processing ability. The special functions like the Differential of One-way Range 
(DOR) and rapid spacecraft fringe search are essential for some navigation missions. As far as the 
correlator output data format is concerned, the FITS-IDI and NGS format will be the standard 
format for geodetic and astrophysical observations; while for navigation, the correlator will use the 
special CVN format. 

To meet the different requirements of CVN observations, we are developing two kinds of software 
correlators. One is for spacecraft navigation; it adopts the Pthreads parallelization method and 
uses the Symmetric Multi Processing (SMP) server as the computation platform. The other 
correlator is for geodesy and astrophysics applications; it adopts the Message Passing Interface 
(MPI) programming and uses the SMP cluster as the platform. Since the GPU (Graphic Processing 
Unit) is powerful for data-parallel processing, it can be used as a computation accelerator in both 
software correlators. 
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Table 1. New CVN correlator requirements and realization. 


Applications 

Geodesy/ Astronomy 

Navigation 

Station 

2-10 

4-8 

Processing speed/station 

>1 Gbps 

16-128 Mbps 

Data transport 

Disk YLBI, e-VLBI 

Real time VLBI, Disk VLBI 

Output format 

FITS-IDI, NGS 

CVN 

Special function 

none 

Rapid fringe search, DOR 

Parallel computing 

MPI + Pthreads 

Pthreads 

Hardware Platform 

SMP server + GPU 

SMP cluster + GPU 


2. SMP Software Correlator 

The SMP software correlator consists of the correlation module, the data preprocessing module, 
the satellite fringe search, and the PCAL abstraction module. It uses Pthreads to realize the 
parallel correlation on any SMP server with x86 family CPUs. It has special functions like the 
rapid spacecraft fringe search and delay model reconstruction and full Phase Calibration (PCAL) 
signal detection. First designed for the Chang’E project and as the main VLBI correlator in the 
CE-1 mission, the SMP software correlator played an irreplaceable role in the CE-1 data processing. 
In the critical flying mission, there were 36 observations (336.55 hours). All of them were processed 
by this software correlator. In the long-term in-orbit operation mission, there were in total 120 
observations (607.39 hours), and 434.02 hours of them were processed by the software correlator. 
Especially in the case of several real-time CE-1 maneuver tracking observations, only the software 
correlator produced the correct results [1]. 

After the CE-1 mission, the performance improvement of this correlator is still being worked on. 
Now the SMP software correlator has been upgraded from 4-station to 10-station capability and 
can output FITS-IDI or NGS format data, besides the CVN format data. Further, the correlator 
data latency can be less than 1 minute now. 


Table 2. Specifications of SMP software correlator. 


Architecture 

FX 

Station number 

1-10 

IF number 

1-16 

Frequency channel 

32-65536/IF 

Input data format 

Mark 5A, Mark 5B 

Output data format 

CVN 

Sampling 

1-bit, 2-bit 

Fringe search 

2-4 stations 

Correlation speed/station 

128 Mbps, 4 stations, 2-bit 

Data turnover 

< 1 minute 

PCAL detection 

Yes 
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Up to now, this correlator has been used in all kinds of experiments. We used the software 
correlator to get the preliminary image of pulsar B0329+54 last July (Figure 3). This is the first 
CVN VLBI image. From 2007 to 2009, CVN carried out several Mars Express (MEX) experiments. 
The SMP software correlator processed all the observation data (Figure 4). For the eclipse of the 
Sun on 22 July 2009, a China- Japan joint experiment was conducted to measure the Total Electron 
Content (TEC) by VLBI and GPS observations. In the CVN data center, it was still the software 
correlator that correlated the VLBI data. The result is under comparison with that of NICT 
[1, 2, 3], 


sb-x3,rate:ps/s sk-x3 



R.A. (J2000) 


Figure 3. Preliminary image of PSR B0329+54. Figure 4. VLBI delay/rate of Mars Express. 


3. High Speed SMP Cluster Software Correlator 

Although the performance of an SMP server is becoming more powerful, the speed of the corre- 
lator on a single SMP server is still strongly restricted. To meet the wide bandwidth scientific VLBI 
observations, the high speed software correlator can work on the fast High Performance Computing 
(HPC) cluster. As the hardware platforms of the high speed prototype software correlator, two 
sets of blade clusters have been installed at the CVN Data Center (Figure 5). One is a 6-node 
cluster with 48 CPU cores, and there is a 10 Gigabit Ethernet (10GE) connection between each 
computation node. Another one is a 16-node cluster with 128 CPU cores, using InfiniBand (IB) 
for data communication. A 10-station prototype software correlation is under development. We 
will use the prototype correlator to compare the performance difference on the two blade clusters. 

Figure 6 shows the structure of the prototype highspeed software correlator. This correlator 
will read the VEX file and the SKD file in the astrophysical or geodetic observation. The Pulsar 
binning function will be added later. The output data format is the standard FITS-IDI or NGS. 
On 6 January 2009, the prototype correlator conducted the first CVN 2-station high speed e-VLBI 
(256 Mbps/station) experiment on the baseline Shanghai-Urumqi. 
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Figure 5. Blade cluster. 



Correlation module 


Output 



Figure 6. Highspeed software correlator structure. 


4. Plans for 2010 

In 2010, with the operation of the CMONC Project, a geodesy and astrophysics oriented 
software will come into operation. We will also utilize the CVN for rapid dUTl measurements and 
rapid e-VLBI imaging experiments. A Graphics Processing Unit (GPU) test bench will be used to 
study the software correlator of CPU + GPU structure. 
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